Structural feature selection for wrapper methods

The wrapper approach to feature selection requires the assessment of several subset alternatives and the selection of the one which is expected to have the lowest generalization error. To tackle this problem, practitioners have often recourse to a search procedure in a very large space of subsets of features aiming to minimize a leave-one-out or more in general a cross-validation criterion. It has been previously discussed in literature, how this practice can lead to a strong bias selection in the case of high dimensionality problems. We propose here an alternative method, inspired by structural identification in model selection, which replaces a single global search by a number of searches into a sequence of nested spaces of features with an increasing number of variables. The paper presents some promising, although preliminary results on several real nonlinear regression problems.